1. Field of the Invention
The present invention relates to an image processing apparatus and image processing method, and a computer program, which can visually remove noise components from image data on which noise components that are not contained in the original signal components are superposed.
2. Background of the Invention
Conventionally, a technique for removing noise components from a digital image on which noise components that are not contained in the original signal components are superposed has been studied. The characteristics of noise to be removed are diverse depending on their generation causes, and noise removal methods suited to those characteristics have been proposed.
For example, when an image input device such as a digital camera, an image scanner, or the like is assumed, noise components are roughly categorized into noise which depends on the input device characteristics of a solid-state image sensing element or the like and input conditions such as an image sensing mode, a scene, or the like, and has already been superposed on a photoelectrically converted analog original signal, and noise which is superposed via various digital signal processes after the analog signal is converted into a digital signal via an A/D converter.
As an example of the former (noise superposed on an analog signal), impulse noise that generates an isolated value to have no correlation with surrounding image signal values, noise resulting from a dark current of the solid-state image sensing element, and the like are known. As an example of the latter (noise superposed during a digital signal process), noise components are amplified simultaneously with signal components when a specific density, color, and the like are emphasized in various correction processes such as gamma correction, gain correction for improving the sensitivity, and the like, thus increasing the noise level.
As an example of deterioration of an image due to noise superposed in a digital signal process, since an encoding process using a JPEG algorithm extracts a plurality of blocks from two-dimensional (2D) image information, and executes orthogonal transformation and quantization for respective blocks, a decoded image suffers block distortion that generates steps at the boundaries of blocks.
In addition to various kinds of noise mentioned above, a cause that especially impairs the image quality is noise (to be referred to as “low-frequency noise” hereinafter) which is generated in a low-frequency range and is conspicuously observed in an image sensed by a digital camera or the like. This low-frequency noise often results from the sensitivity of a CCD or CMOS sensor as a solid-state image sensing element. In an image sensing scene such as a dark scene with a low signal level, a shadowy scene, or the like, low-frequency noise is often emphasized due to gain correction that raises signal components irrespective of a poor S/N ratio.
Furthermore, the element sensitivity of the solid-state image sensing element depends on its chip area. Hence, in a digital camera which has a large number of pixels within a small area, the amount of light per unit pixel consequently decreases, and the sensitivity decreases, thus producing low-frequency noise. For example, low-frequency noise is often visually recognized as a pseudo mottled texture across several to ten-odd pixels on a portion such as a sheet of blue sky or the like which scarcely has any change in density (to be referred to as a “flat portion” hereinafter). Some digital cameras often produce false colors.
As a conventionally proposed noise removal method, a method using a median filter and a method using a low-pass filter (to be abbreviated as “LPF” hereinafter) that passes only a low-frequency range have prevailed.
In the method of using a median filter, a pixel value which assumes a central value (to be referred to as “median” hereinafter) is extracted from a region (to be referred to as “window” hereinafter) which includes a pixel of interest and its surrounding pixels, and the extracted median replaces the pixel value of interest. For example, many methods using a median filter such as Japanese Patent Laid-Open No. 4-235472 and the like have been proposed. Especially, when the pixel of interest is impulse noise or random noise, the median filter replaces the pixel value of interest as an isolated value having a low correlation with surrounding pixels by a median value with a high correlation to surrounding pixels, thereby removing the isolated value in original image information.
On the other hand, in the method using an LPF, the average value of a plurality of pixels around a pixel of interest to have the pixel of interest as the center is calculated, and replaces the pixel value of interest. FIG. 19 shows an example of a conventional LPF which calculates the average value of a plurality of pixels around a pixel of interest to have the pixel of interest as the center. This method is mainly effective for block distortion mentioned above. That is, since block distortion is noise that generates block-like steps different from signal components on a portion which is originally a flat portion, the steps can become hardly visible by moderating the gradients of the steps.
The aforementioned two noise removal methods can effectively work locally, but have adverse effects (blur of an edge portion and the like). Hence, many modifications of these methods have been proposed. For example, Japanese Patent Laid-Open No. 2001-245179 discloses a method of making product sum calculations by selecting only surrounding pixels which are approximate to the pixel value of interest upon calculating the average value, so as not to blur an image due to a noise removal filter process.
In addition to the above two methods, i.e., the method of using a median filter and the method using an LPF, many other methods that pertain to noise and distortion removal have been proposed. For example, Japanese Patent Laid-Open No. 8-56357 discloses a method of replacing signal values between pixels which are located on the two sides of the block boundary, so as to remove block distortion. Furthermore, Japanese Patent Laid-Open No. 10-98722 discloses a method of adding a predetermined pattern selected from a plurality of patterns based on a random number to pixel signal levels around the block boundary.
Moreover, Japanese Patent Laid-Open No. 7-75103 discloses a method of removing block distortion produced upon encoding by adding an error to the level value of a specific pixel of interest having the block boundary as the center. In addition, Japanese Patent Laid-Open No. 4-239886 discloses a method of removing noise by detecting maximum and minimum values from pixels which neighbor the pixel of interest, and selecting one of the maximum value, the minimum value, and the pixel value of interest using, as a control signal, the determination result indicating whether or not noise is contained therein, so as to remove white and black points having isolated values.
However, none of these conventional methods can exhibit a perfect noise removal effect of the aforementioned low-frequency noise. For example, the method using a median filter has only an effect of deleting an isolated value which has a low correlation with surrounding values, and the method using an LPF is effective only for high-frequency noise or white noise with high randomness by cutting off a high-frequency range. Hence, these methods are not effective for low-frequency noise, and the low-frequency noise remains unremoved.
The method described in Japanese Patent Laid-Open No. 8-56357 or the like, which aims at removing block distortion, can effectively reduce the deleterious effects of the steps by a method based on random number addition, a method of replacing pixel values between blocks or the like, as long as the block boundary is known, since block distortion to be removed is a high-frequency component generated as a rectangular step. However, the low-frequency noise to be removed is connectivity noise, i.e., pixel values which have fewer changes that successively appear across a broad range from several to ten-odd pixels, and the aforementioned technique that reduces block distortion cannot be directly applied. Of course, the generation position of noise is not known, unlike the block boundaries upon block encoding.
On the other hand, the method based on random number addition applies a pixel value which is not present in surrounding pixels. Hence, especially in a color image, when random numbers are added to respective color components obtained by color separation, a new color which is not present in surrounding pixels is generated, and deterioration of image quality, due to, for example, the generation of false colors or the like occurs.
Japanese Patent Laid-Open No. 7-203210 discloses an invention that reduces the signal strength of a specific frequency, although its object is different from noise removal of the present invention. In a system which receives a halftone dot document and executes dithering of a pseudo halftone process upon output, moiré is generated due to interference of the frequency of the input halftone dots and that of dithering. The invention described in Japanese Patent Laid-Open No. 7-203210 is a moiré removal method that removes the frequency of the input halftone dots to prevent the generation of moiré.
That is, this moiré removal method replaces the pixel value of interest by a pixel value which is located ahead of the pixel of interest by a distance corresponding to a predetermined number of pixels on a line, since it is effective to disturb a certain regularity exhibited by the image to change the frequency of halftone dots. The above invention discloses cases wherein the predetermined number of pixels is fixed and is randomly selected.
However, since this moiré removal method aims at disturbing a specific period corresponding to a peak, it is not completely effective for low-frequency noise, which is generated in a broad low-frequency range. Since this method replaces pixel values, density preservation is guaranteed, but that method is merely a process of changing the selected pixel value, i.e., spatially shifting pixel phases.
Furthermore, since a change in a selected pixel value corresponds to cyclic filter characteristics, an impulse response becomes infinity. Even when the predetermined number of pixels as the distance between pixels to be replaced is randomly selected, since sampled pixels are replaced in turn, the moiré period is merely displaced by shifting peak phases of halftone dots, which are generated at specific periods.
As described above, none of the aforementioned prior art can effectively remove low-frequency noise components contained in image data.